System Understanding and Statistical Uncertainty Bounds from Limited Test Data

n many DoD test and evaluation programs, it is necessary to obtain statistical estimates for parameters in the system under study. For these estimates to provide meaningful system understanding, uncertainty bounds (e.g., statistical confidence intervals) must be attached to the estimates. Current methods for constructing uncertainty bounds are almost all based on theory that assumes a large amount of test data. Such methods are not justified in many realistic testing environments where only a limited amount of data is available. This article presents a new method for constructing uncertainty bounds for a broad class of statistical estimation procedures when faced with only a limited amount of data. The approach is illustrated on a problem motivated by a Navy program related to missile accuracy, where each test is very expensive. This example will illustrate how the small-sample approach is able to obtain more information from the limited sample than traditional approaches such as asymptotic approximations and the bootstrap. (